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Abstract 

A poorly designed knowledge base can be as^cryptic aaa'n arbitrary program and just as difficult to 
maintain. Representing control knowledge "abstractly, separately from domain facts and relations, 
makes the design more transparent and explainable. A body of abstract control knowledge, provides 
a generic framework for constructing knowledge bases for related problems in other domains and 
, lalso prcfira^a useful starting poiptfo^tudying the nature of strategies. * . ^ 

1: Introduction v • * 1 { 

The quality of a knowledge base depends not only on how well it' solves problems, £ut also how on j 
easily its design allows it to be maintained. Easy maintenance-the capability to reliably modify a 
knowled^e^se without extensive reprogramming-is important for several reasons: 

• Knowledge-based programs are°byilt incrementally, based on many trials, so modification r 
is continually required, including updates based on imprdved expertise; ' , 

• V ■ ' - ' ' 

• A knowledge base is a repository that other researchers and users may wish to build ' 
upon yedrs later; ; ' • * \ - 

• A client receiving a knowledge base constructed for him may wish to correct and exteijd 
it without tye assistance of the, original'designers. . 

- : \\ , ' ; : ; : / , ^ . *' > . ; /• v 

A knowledge base is like a traditi6naf program in that maintaining It requires having a good 

•understanding of the underlying design. That is, you nfeed/KS'know how the parts of the knowledge 

base are expected to interact in problem solving. Depending on the representation, this includes 

knowing how default and judgmental knowledge interact, whether rule clauses can be reordered, 

** \ - ***** • 

when attached procedures are &pplied t Tipw constraints ate inherited and ordered, etc. One way to, 

provide this understanding is to. have the program ekplain its reasoning, using an internal description 

of its own design {Davis, 1976), (Swartout, 1977). Howevfer, problems encountered in understanding 

traditional prQgran>6;-poorty : structurec^.codei' implicit side-effects, ahb inadequate documentation- - 

carry over to knowledge-based ^programming and naturally limit Jhe capabilities of explanation 

programs. For example, a knowledge b&se might arbitrarily combine reasoning strategies with facts 

about the domain. Implicit, procedurally-embeddgd knowledge cannot be articulated by an 

explanation systerfw (Swartout, ■ 1*081)* (Clancey, 1983) ancf is not visible to guide the program 

maintainer (seeJEnhjsf, 1982) for an entertaining studyi>f this problem). . ' ' . *t ■ 

This paper argues that ah important design principle for building expert Systerhs is to represent all 
control knowledge abstractly, separata from the domain knowledge it operates upon. This idea i5 
illustrated with examples from the NEOMYCIN system (Clancey, 1981). There are many scientific, 



engineering, practical benefits. The difficulty of attaining this^dead design is also considered.' 

2. What is Abstract Control Knowledge? 

"Control' knowledge" Specifies when and how a program is to carry out its operations, such as 
pursuing a goal, focusing, acquiring data, and making inferences. A basic distinction can be^made 
between the faqts and relations of a knowledge base and the program operations that act upon it. For 
Example, facts and relations in a medical knowledge base might include (expressed in a predicate 

calculus formulation): » . 

\ • - * 

* u . 

(SUBfYPE INFECTION MENINGITIS) . / 

— "meningitis ij5 a kind of infection" 

(CAUSES INFECTION FEVER} , X ■ f 

— "infection causesjfey^r" x ■ 

(causes Infection ,shaking-chills) , . • • • 

— "infectipn causes shaking- c;h ill s" f , 

•^DISORDER .MENINGITIS) y. ' m 

"meningitis is a disorder", 

"(RINDING FEVER) t , , * . ' \ ? v * * < ' 

— "fever is a finding", ' 1 ' , r . 

* ' . %' * * * 

SUch a knowledge b$se might be used tp provide consultative advice to a user, in away typicalof 
-expert systems (Quda, and" Shojlliffe, 1983). Consider, for example, a consultation system for 
diagnostog some faulty device. Otaftypical program operation is to select a finding that tauses a* 
disorder and ask the user to indicate whether ihe device being diagnosed exhibits that symptom. 
Specifically; af medical diagnostic system might ask the user whether thef fetfent is suffering from 
shaking chills, in order to determipe .whether he has an infectifin. ' The first description of the 
program's operation is abstract, referring only la domain-independent relations like "finding" and 
"causes"; the second description is concrete, referring^) domain-dependent terms like "shaking- 
chills" and "infection"/ ("Domain-independent" doesn't mean that it applies to every domain, just 
that the term is not specific to any ope domain.) , 

-The operation described here can be characterized abstractly as "attempting to confirm a 
* diagnostic hypothesis" or'-concr^tely as "attempting to determine whether the patient has an 
. infection." Either description indicates thp strategy that motivates the question the program is asking 
of the user. So in this example we see how a strategy, or contrQl knowledge, can be stated either 
abstractly or corKCretely. The following two examples illustrate how both forms of control knowledge 
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' f might be represented in a knowledge baste. 




2.1JVn Implicit Refinement Strategy * 

In MYCIN (Shortliffe, 1976) r most knowledge is represented as domain-specific rules. For exartit 
the rule "If the patient h^s an infection and his CSF cell coiknt is leSs than 10, then it is unlikely that he 
has meningitis," might be represented as: . ^ 

PREMISE: 

\($AND (SAME CNTXT INFECTION) , /» 

... (IVESSP (VtfLl CNTXT CSFCELLCOUNT) 10)) 1 ■* 

ACTION: * • 

- (CONCLUDE CNTXT INFEtYlON^fPE MENINGITIS TALLY -700) 

, : ■ / ; ' ; .4 / !■ 

The order of Clauses ^important here, for the program should not consider the "CSF cell count- if r~ j 
the patient does not havp an infection. Such clause ordering in all rules ensures that the program 
proceeds by top-down refinement from infection to meningitis to subtypes of meningitis. The disease 
hierarchy cannot be stated explicitly in the MYCIN rule language; it is implicit in the design of'the . . , 
rules. (See (Clancey, 1983) for further analysis of the limitations of MYCIN'S representation.) J 

\ "-■ • /• . .. ' ± 

CENTAUR (Aikins, 1980) is a system in which disease hierarchies are explicit. In its representation 
language, MYCIN'S meningitis knowledge might be encoded as follows (using a LISP property list 
notation): s 4 \ 

' V : / i ^ 

'; INFECTION .„ \ ' J' - 

MORE-SPECLFJC ((disease MENINGITIS) 

V\. v (disease BACTEREMIA)...) 
IF-CONFIRMED^ -'- ' ^DETERMINE disease of INFECTION) 

f 

MENINGITIS 

• MORE-SPECIFIC ((sfibtype BACTERIAL) , ■ 

(subtype VIRAL)...) 
IF-CONFIRMED (DETERMINE subtype of MENINGITIS) 



In CENTAUR, hierarchicatf^elations among disorders are explicit (meningitis is a specific kind of 
infection), and the strategies for using the knowledge are domain-specific (after confirming that the 
patient has an infection, determine what more specific disease he has). This design enables 
CENTAUR to articulate its operations better than MYCIN, whose hierarchical relations and strategy, 
are procedurally embedded in rules. 

However, observe that each ncxte of CENTAUR'S hierarchy essentially repeats a single strategy-try 



to confirm the presence o^a child disorder-and the, overall strategy of top-down refinement is not 
explicit. Aikins has labeled CENTAUR'S strategies, but has not stated them abstractly. By. N 
Representing strategies abstractly, it is possible to have a more explicit and non-redundant* design. 

• This is what is done in NEOMYCIN. 

• % 

'in NEOMYCIN domain relations and strategy are represented separately and strategy is. 
represented abstractly. A typical 4 rule that accomplishes, in part, the abstract task of attempting to 
confirm a diagnostic hypothesis and its subtypes is shown below. / ^ 

' <Domai^Knowledge> ■ ■* m . 

INFECTION 

CAUSAL-SUBTYPES (MENINGITIS BACTEREMIA ...) 

• r. . > , 

MENINGITIS ' 

CAUSAL-SUBTYPES (BACTERIAL VIRAL ,..) ] ^< 

<Abstract Control Knowledge> 

TASK: EXPLORE -AND -REFtNE ' • ■ 

ARGUMENT: CURRENT -HYPOTHESIS 

METARULEuOl ■ # , . . , ' 

IF the hypothesis being focused upon . > 

has J a child " 

that has not <been pursued, '. ■ ' 
"WEN pursue that child. \ _ 

(IJ\ (AND (CURRENT-ARGUMENT SCURFOCUS) 

(CHRDOF SCURFOCUS SCHILD) ' * . * 

(THNOT (PURSUED SCHILD) t) ' . • ■ 

(NEXTACTION ( PURSUE-HYPOTHESIS SCHILD))) . * . . 



NEOMYCIN, uses a deliberation/action loop, for deducing what it should do next. Metarules, like 
the one shown above, recommend what tfefk shoulttise done. next, wha) domain rule applied, or what 
domain finding requested from the User (details are given in (Clancey! 1881) and (Clancey and Bock, 
1982) and are not important here). The important thing^to notice is that this metarule will be applied . 
for refining any disorder, obviating the need to "cqmpile" redundantly'into the domain hierarchy oM 
disorders how it should be searched. When a new domain relation is declared, (e.g., a new kind of 
infection is added to the hierarchy) the abstract control knowledge will use.it appropriately. That is, 
we separate out what the domain knowledge is from how if should be used. 

". ; # . ■ . ' 

Metarules were first introduced for use in expert systenmsiby Davis (Davis, 1076), but he conceived 
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of them as being domain-specific. In that form, principles are encoded redundantly, just tike y , 
CENTAUR'S control knowledge. For example, the principle Qf pursuing common causes before' 
\jnusual causes appears a^s specific metarules for ordering the domain rules of each disorder. 

- The benefits of stating metarules abstractly are illustrated further by a sepond example. % 

2.2. An Irtiplilit Question-Asking Strategy 

Another reason for ordering clauses in a system like MYCIN is to prevent unnecessary requests for 
data. A finding might be deduced or ruled out from -other facts available to the program* For 
* example, the rule "If the patient has undergone surgery and neurosurgery, then consider diplococcus • 
as a cause of the meningitis 11 might be represented as follows. 

PREMISE : (SAND (SAME CNTXT SURGERY) : 

(SAME ,CNTXt NEUROSURGERY)) 
ACTION: (CONCLUDE ClftXT COVERFOR DIPLOCOCCUS TALLY 400) 



We^ay that the surgery clause "screens" for the relevance of asking about neurosurgery. Observe 

that neither the relation between these tv^o Findings (that neurosurgery is a type of surgery) nor the 

strategy of considering a geh^fal finding in order to rule out one of its subtypes is explicit. An 

amative way used in MY&N for encoding this knowledge is to have a separate "screening" rule 

that at^least makes dear thatJv6se*two findings are related: "If the patient has not undergone 

surgery, then^ieJigsnotufWergone neurosurgery" 

* "'■ ' V * 

PREMISE: (SAND (NOTSAME CNTXT SURGERY)) v 
ACTION: (CONCLUDE CNTXT NEUROSURGERY YES TALLY -1000) 



Such a rule obviates the need for a "surgery" clause in every rule that mentions neurosurgery, so w 
this design is more elegant and less prone to error. However, the question-ordering strategy and the 
abstract relation between the findings are still not explicit. Consequently, the program's explanation 
system cartnot help a system maintainer understand the underlying design. 



In NEOMYCIN, the above rule is represented abstractly by a metarule for the task of finding out new 
data Y * / 



<DomairtKnowledgo> % * 

***..'. » * . • 

("SUBSUMES SURGERY NEUROSURGERY) ... ' .. 

(SUBSUMES SURGERY CARDIACSURGERY) . ' 

v 1 ■' '. 

<Abstract Control Knowledge> * 



TASK: FINDOUT ' / 

ARGUMENT: DESIRED-FINDING. | ' '* < 

METARULE002 . ' 

IF th» desired finding ' 

is a subtype of a class of findings 1 and • ' - 

the class of findings is not present in this case, 
* THEN conclude that "the desired rinding is not present.' 

(IF (AND (CURRENT-ARGUMENT SSUBTYPE)' \ —} 

(SUBSUMES SCLASS SSUBTYPE) ' / , . 

(THNOT ( SAME P CNTXT SCLASS))) ^ - v '*-•'* 

' (NEXTACTlON ' ' . •*•... ' 

(CONCLUDE CNTXT SSUBTYPE 'YES TALLY -1000))) ° 

ThismetarJe is really an abstract generalization of ^screening rules. Factoring out the statement 
of relations among findings from how those relations are to be used produces an elegant and 
( economical representation. Besides enabling more-detailed explanation, such a design makes the 
system easier to construct and more robust. 

Consider the multiple ways in which a single relation between findings can be used. If we are fold 
that the patient has neurosurgery, we can use the subsumption link (or 4s inverse) to conclude that 

^ the patient has undergone surgery. Or if. we know that -the patient has not undergone any kind of \ 
surgery we know about, we can use the "closed world assumption" and conclude that the patient has 

s not undergone surgery/ These inferences are controlled by abstract metarules in NEOMYCIN. 

The knowledge base is easier to construct because the expert needn't specify ^fry situation in 
which a given fact or relation should be used. New facts and relations can be added in a simple wky; 
the abstract metarules explicitly state how the relations will be used. The same generality makeshift 
knowledge base more robust. The system is capable of making use,of facts and relations for deferent 
purposes, perhaps in combinations that would be difficult to anticipate or enumerate. i* ■ o • 



3. Studying Abstract Strategies and S^ v u*ctu raj Relations >" . * 

Iri NEOMYCIN, domain findings and disorders are related In. the way shown above, and' there are 
approximately 75 metarules-that constitute a procedure for doing diagnoses. Besides abstract domain 
relations, such as SUBSUMES, NEOMYCIN'S metarules reference: 

. ' : . v. : • • 7 . ; \ • *"\ , ' " •• 

• Knowledge about metarules and tafeks: (static)* the argurcpnt of a task, whether metarules 
are to be appl^ iterati^ which a task should be aborted, 

: "(dynamic) whetter £ tatek completed Successfully, whether metarule -succeeded^or 

Mailed.etc. * * ■ ✓ - V; », • ' 

> Domain problem-solving history: the activje hypotheses) whethter a hypothecs was 
pursued, cumulative belief for a hypothesis rules using a finding thptare "in; focus", a 
strong competitor to a given hypothesis; etcV. r : , • * 

These concepts form the vocabulary for a model ofrdiagrtosis, the terms in which expert behavior is 
interpreted and strategies are expressed. ^ v . * ■ • T 

• 3 . , • ; ; 

An unexpected effect is that there is rib rfibre backward chaining at the domain level.^Thatjs, the 
only reason MYCIN does backward chaining during its-diagnostic (histoi^^ is to 

accomplish JtopdowlT refinement and to apply "screening rules. This is an important result. By 
studying the hundreds of rules in the mVCIN systerfi, factoring out domain relations from control 
knowledge, we haya-greatly deepened oyr unjjscstanding of the knpwtedjje encoded in the rules. 
There arejyvo specific products: a body abstract control knowledge that can itself be studied, as 
well as applied in other -problem domains! and a . language x for representing knowledge about 
disorders (in terms of causality, subtype, etc.). We call these abstract rei^Wofts structural relations, 

■ '•• /; . V : •". .'• ' ■ 

Structural relations are a means for indexing domain*specifrc knowledge: They select hypotheses 
to focus uport, findings to request, and domain inferences that might be made. As such, structural 
relations constitute the organization, the acces^ paths, by which ^strategies luring domain-specific 
knowledge into plajg_ For example, the metarules given above mention the CHH-DOF-and SUBSUMES 
relations. METARULE001 looks for the* children 6f the current hypothesis in order to pursue them; 
METARULE002 looks for a more general finding in-order to ask for it first. 

■ " 7 . . ■■"■V ■ t / f /f '■ 7- 4 

The$e relations constitute the language by which the primitive domain concepts (pkrticulai^findings 
and disorder hypotheses) are related in a network. Adding a pew strategy of fen requires adding % 
*new kind of structural relation to the network^or example, suppose we desire to pursue common 
causes of a disorder ^f ore serious, but unusual causes. We must partition the causes of any 
disorder according to this distinction, adding new relations to our language--COMMON-CAU^ES and 



; SERIOUS-CAUSES; , . 

~>- • -V V .. . 

♦ Similarly, frte applicability of a strategy clef 
domain. For exampte, a strategy mijpht gii 
problem domain all findings might be equall 
with how to^arch^deei^terarchy"ofidi 



shallow, making the strategies inapplicabl 

.. : . :> - • • » ■ % 

explicate structural relations. On this basis > 
of strategies, referring to structural propertte 

Lepat has found a similar relationship betv 
N in his program for discovering new heuristic 
heuristics in EURISKO depends on breaking 
slot^ that the program cart inspect and mo< 
-concepts /Mfrhbse Representation is refined 
\ FACTORS slot' to every.fyjmber).* -The* prog 

that collects relatibns'among eritriesof an in 

— - . .. v . ^ . x * 

4. Given. the Beqefrts,' Cantit 

An initial reaction might*be that for some 
abstract straiegies-all facts and relations a 
tha-procedure for confirming any given df$ 
some device) mightjge completely situation 
w6uld appear to be air unusual kind of ;doir 
principles can be applied over and over aga 

Teaching and learning are made incredib 
problem to another! Domains with a StrtJ 
might be like this. Perceptual skills rely on \ 
of xlaia; they are might be poor candidates f 

•We also know that in many domains, for 
■ •> • • 

solving routine problems. These prctcedur 

for organization management, equipment^ 

inaportant to iecaflpize that these procedui 
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causal, temporal, and spaciaMjiteractionS, problem-solving goals, abstract principles of design, 
diagnosis, fitc. Except where a procedure is arbitrary, there must be some underlying rationale for 
the selection and ordering of its steps. Knowing this rationale is certainly, important for reliably 
modifying' the procedure; such procedures are often just prepared plans that an expert (or -a user 
following a program'? advice) may need to adapt to unusual circumstances. At one level, the , 
rationale can be made explicit in terms of an abstract plan with its attendant domaifi structural 
relations; a redundant, compiled form 6arv.be used for efficient routine problem solving. 

In theory, if the rationale for a procedure or prepared plan can be made explicit, a program can 
reconstruct the procedure from first principles. This approach has two basic difficulties. First, the 
procedure might have been learned incrementally from case experience. It simply handles problems 
well; there is no compired-out theory that can be articulated. This problem arises particularly for skills 
in which behavior has been shaped over time, or for any problem in which the trace of "lessons" has 
been poorly recorded. The second difficulty, is that constructing a procedure from first principles can 
involve a great deal of search. Stefik's (Stefik, 1980) multi-leveled planning regime for constructing 
MOLGEN experiments testifies to the complexity of the task and the limited capabilities of current 
programs. In contrast, Friedland's (Friediand, 1979) approach of constructing experiment plans from 
skeletal, abstract plans trades flexibility for efficiency and resemblance to human solutions. While 
skeletal plans may sometimes use domain-spetific terms, as precompiled abstract procedures they 
are analogous to NEOMYCIN'S tasks. * 4 % - 

Importantly, the rationale for the abstract plan itself is not explicit ih any of these programs. For 
example, NEOMYCIN'S metarules for a given tstsk might be ordered by preference (alternative 
methods to accomplish the same operation) or as steps in a procedure. Since the constraints that 

^suggest the given ordering are not explicit, part of the design of the program is still not explicit. For 

* i • 

example, the abstract steps of top-down refinement are now stated, but the sense in which they 

constitute this procedure isjiot represented. (Why should pursuing siblings of d hypothesis be done 

* « 
before pursuing children?) <As*another example, the task'of "establishing the hypothesis space" by 

expanding the set of possibilities beyond fcommon, expected caused and then narrowing down in a 

refinement phase has mathematical, set-theoretic Underpinnings that are not explicit in the program. 

Similarly, Stefik's abstract planning procedure of "least-commitment" is implicit in numetfcfprioritiea 

assigned to plan design operators (Clancey, 1983). Automatically constructing procedures at this 

high level of Abstraction, as opposed to implicitly building them into a program, has been explored 

very little. \ t ■ <.'•'* 

Even within the, practical bounds of what we make explicit, it might be argued that representing 
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V < v ; 

procedures abstractly is much more difficult than stating individual situation specific rules. This 
might differ from person to person; certainly in medicine §ome physicians are better than others at 
stating how they reason abstractly: A good heuristic might be to work with good teachers, for they 
are most likelylo have extracted the principles so they can be taught to students. 

■*<-.' * . * * 

* There is certainly an initial cost whose benefit is unlikely to be realized if no explanation facility is 
desired, only "the original designers maintain or modify the knowledge base, or there is no desire to 
ild a generic system. But even this argument is dubitatole: a knowledge base with embedded 
strategies can appear cryptic to even the original designers after it .has been left aside for a few< 
months J Alsov anyone intending to build more than one system will benefit from expressing 
knowledge ^ generally as possible so that lessons about structure and strategy can speed up the 
building of hew systems. 

The cost aside, it appears that there is no way to get strategic explanations without making domain 
relations expjicit and stating strategies separately. This was the conclusion of Swartout, who was Jed , 
to conclude that an automatic programming approach* as difficult as it first seemed, was a natural, 
direct way to ensure that the program had knowledge of its own design (Swartout, 1981). That is, 
providing complete explanations means understanding the design well enough to derive the 
procedures yourself. ■" 

NEOMYpiN's factoring of knowledge into domain and strategic knowledge bases is comparable to 
the input requirements of Swartoufs automatic programming system. However, NEOMYCIN 
Interprets* its domain knowledge, rather than instantiating its abstract strategies in a compiled 
program, (Maintaining the separation is important so the metarules can be used in student modeling 
(London arid Clancey, 1982).) Moreover, NEOMYCIN'S strategies are abstract, unlike the domain- 
specific "principles" used in Swartoufs program. This design decision was originally motivated by 
our desird to replicate the kind of explanations given by teafthera(HasHng, 1983). However, we now 
realize that representing control knowledge abstractly has engineering and scientific benefits as well. 

5. Advantages of the Approach 

The- advantages of. representing control knowledge abstractly can be summarized according to 
/ engineering, scientific, and practical benefits: 

^Engineering. ^ 

/: . o The explicit design is easier to debug and modify. Hierarchical relations among 
I findings and hypotheses and search strategies are no longer procedurally 



■ / 

* / 

/ 



11 



v embedded in rules. < 

, o Knowledge is represented more generally, *so wd get more performance from less 
system-building effort. W9 don't need to.specify every situation in which a given 

fact should be used. > • • •'• 

♦ * 

^oThe body bf abstract control knowledge can be applied to other problems, '. 

constituting the basis of a generic system,' for example, a tool for buildir\g 
„ consultation programs thai dQ diagnosis. ^ > 

• Science. Factoring out control knowledge from domain knowledge provides a basis for 
studying the nature of strategies. Patterns become clear, revealing, for example, the 
underlying structural bases for backward chaining. Comparisons between domains can 
be made according to whether a given relation exists or a strategy can be applied. 

• Practice. 

o A considerable savings in storage is achieved if abstract strategies are available for 
solving problems. Domain-specific procedures for dealing with all possible 
situations need|4 be compiled in advance. 

o Explanations can be more detailed, down to the level of abstract relations and 
strategies, so the program can be evaluated more thoroughly and used more 
responsibly. 

6 Because strategies are stated abstractly, the program can recognize the 

7 application of a particular strategy in different situations. This provides a basis for 
explanation by analogy, as well as recognizing plans duringjjnowledge acquisition 
or student modelling. ' 

Representing control knowledge abstractly moves us closer to our ideal of specifying to a-program 

WHAT problem to solve versus 'HOW to solve the problem (Feigenbaum, 1977). Constructing a 

1 * , 

knowledge base becomes a matter of declaring knowledge relations. HOW the knowledge will be 
used needn't be simultaneously and redundantly specified. ^ 

An analogy can be made with GUIDON (Clancey, 1979) (Clancey, 1982), whose body of abstract 
teaching rules make the program usable with multiple domains. Traditional CAI programs are specific 
to particular problems (not just problem domains) and have both subject matter expertise and 
teaching strategies embedded within them. The separation of these in GUIDON, and now the abstract 
representation^ strategies in NEOMYCIN, is part of the logical progression of expert systems 
research that began with separation of the interpreter from the knowledge base in MYCIN. The trend 
throughout has been to state domain-specific knowledge more declaratively and to generalize the 
procedures that control its application. 
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Another analogy can be made with database systems that combine relational networks with iogicd 
programming (e.g., see (Nicolas. 1977)). To conserve space, it is ndt practical to explicit!^ Storaevery 
^relation amon&pntities in a database. For example, a database j^bdut a population of a^ountry might 
' record just the pai^tsof each person (e.g., (MOTHEROF $CHILD SMOTHER) and jFATHEROF_ 
$CHILD $FATHER)). A sep&^tebody of general derivation axioms is used to retrieye other relations 
(the intensions! database). For example, siblings can be Computed by the rule: 
- ' •: | " / * 

(IF (AND (PERSON SPERSOfl) ^ 
(MOTHEROF SPERSON SMOTHER) 
(PERSON $PER$0»fe) 
(MOTHEROF $PERS0N2 SMOTHER)) 
(SIBLING SPERSON SPERS0N2)) * > 



Such a rule is quite similar to the abstract metarules that NEOMYCIN uses for deducinjlthe 
presence or absence of findings. NEOMYCIN differs from database systems in that its rules are 
grouped and controlled tfc-apfcomplish abstract tasks. Only a fpw of NEOMYCIN'S metarules make 
inferences about database relations; most invoke other tasks/such as "ask a general question" and 
"group and differentiate hypotheses." Moreover, the kho^/ledge base Contains judgmental rules of 
evidence for the disorder hypotheses. These differShcjes aside, the analogy is' stimulating. 'It- 




suggests that treating a knowledge base as ah object to be inspected, -reasoned about, and 
manipulated by abstrapt procedures*^ a database is checked for integrity, queried, and extended by 
general axioms-is a powerful design principle for building expert systems. \, 



Acknowledgments . ^ 

This research has been supported in part by ONR an3 ARI Contract N00014-79C-0302. 
Computational resources have been provided by the SUMEX-AIM facility (NIH grant RR00785). 



16 



References 



\ 



Aikin^J. S . Representation of control knowledge In expert systehis, \n Proceedings of the First AAAI, 



pages 121 123, 1980. 



v I 



Clancey, W. Jl^ Tutoring rules for guiding a case method dialogue. 'The International Journal tif Man- 
. Machine Studies, K \97§, 77, 25-48. " (Also in Sleeman and, Brown (edjtors), Intelligent -Tutoring 
Systems, Academic Press, 1982). m . ' * '» ' . ■ 

Clancey, W. J. arid Letsinger,^ 4 . NEOMYCIN: Reconfiguring a rule-based 'expert system for, 
4/ application to teaching, iri Proceedings/of. the Seventh /JCA/,' pages 829-836, 1981, (Revised 
version to appear in Clancey and Sho^liffe (editors), Readings iri npedical artificial intelligence: 
The first decade, Addison-Wesle>^83). 



9 ■ 

'.■f : 



Clancey T .W» J. GUIDON. In Barr and Feigenbaum (editbrs), The Handbook of Artificial Intelligence, 
chapter Applications-oriented Al research: Education. WilliamiKaufmarin, Inc., Los Altos, 1982. 
■ K ^Revised version to appea* in the Journal of Computer-Based fri&trubtion, 1983). 

Clancey, W. J. The epistemology of a rule-based expert system: AjframQwork for explanation. Artificial 
, Intelligence, 1983, 20(3), 215-251. (Also to appear Jn BucHanan and Shortliffe (editors), 
* . Rule-based expert systems: The MYCIN experiments of the Stanford Heuristic Programming 
Project, AddisorvWesley, 1983). * ■" "i' v* • 

- ' . 

Gtencey, W. J. and Bock, C. MRS/NEOMYCIN: . Representing metabontrolin predicate calculus. HPP 
^ ; Memo 82-31, Stanford University;November 1982^ 

Davte R. Apppci^ns of meta-level knowledge to the construh^n^-ipaintenance, and use of large, 
knowledge bases. HPP Memo 76-7 and Al Memo 283, Stanford University, July 1976. 



Duda, R. O. and Shortliffe, E. H. Expert systems research. Sc/ence^ 1$i83, 220, 261-268. 

Ennis, S. P.Expertispstems: A user's perspective of some current tools* in Proceedings of the Second 
AAA/, pages 31 9-321, August, 1982. * -O-j ■. ■■ * 

Feigenbaum, E. A. The art of artificial intelligence: I. Thertje£:;$nd case studies of knowledge 
engineering; in Proceedings qf the Fifth UCAl, pages 3014-1029, August, 1977. 

Friedland, P. Knowledge-based experiment design in molecular genetics, in Proceedings of the Sixtp 
UCAl, "pages 285-287, 1979. * -^ : 1^V V — 

■' *.. -J- * 

Hasling, D. W. Abstract explanations of strategy in a diagnostic consultation system. (To appear in the 



Proceedings ofAAAI-83). 



Lenat, D. B. The nature of heuristics. Artificial Intelligence, 1982, 19(2), 189-249. 

• *** 

London, B. and Clancey, W. J. Plan recognition strategies in 1 student modeling: prediction and 



description^ Proceedings of the Second AAAI % pages 335-338, >1 982. 

' ■ m c 

Nicolas, J. M. and Gallaire, H. Data base: Theory vs. interpretation. Irj H. Gallaire ^nd J. Minker 

^ (editors), Logic and data bases, pages 33-54. /Plenum Press, New York, 1977. 

: - • ' , , / 

Shortlif f e, E. H.Vomputer^baspd medical consultation's: MYCIN. New. York: Elsevier 1 976. . 

• ^ ' . : » v , . 1 > - 

Stefik, M. J. Planning with constraints. PhD thesis, Computer Science Department, Stanford 
University, 1980.' ws ' , 

Swartout, W. R. A digitalis tfierapy advisor with explanations. Technical report 1 76, MIT Laboratory for 
Computer Science, Febru&jy1 977. ^ 

Swartout W. R. Explaining and' justifying /p expert consulting programs, jn Proceedings of the 
Seventh IJQAI, August, ,19§1. (Also to .appear in Clancey and Shortliffe (editors), Readings in 
medical artificial intelligence: 7Y?af/rsfcfe^acfei Addlsbn-Wesley, 1983). - * 



Hevy - * 

1 DoUrc Ahlere 
Code H7H 

Hunan 'Tac tore Labor ecory 
HAVTtXEOjOIPCES 
Orlando, PL 32B13 

1 Or. Ci liken. 
Navy Paraoanel TtiO Center 
Sia Diego, CA 92l52t • 

1 Dr. Mnryi S. kbr » .•. 

Navy Pereoooel t&O Canter 
N^aJl««o, jCA 92132 

1 Or. lobar e tlauchard . 
Navy Periounel UD Center 
Sao Diego, CA 92132 

1 Liaison Scleotlet 
Off Lea of Naval taaaareb 

trench Qfflce, London 
So* 39 

PPO Nov York, NT 09310 

1 Dr. ti chard Caotont 
Navy taaaareb Laboratory 
Coda 7310 

Washington, DC 20373 

1 Dr.. Stanley Collyar 
Office of NavaJL Technology 
800 N. Quiecy Street 
Arlington, VA 22217 

1 CDt Hike Cur ran 
Of flea of Naval laaaareh 
BOO 8. Quiney Sc. 
Coda 270 
• Arlington, VA 22217 

HJr , 

1 Dr. Juda franklin 
Coda 7310 

Navy taaaareb Laboratory 
Washington, DC 20373 

1 Dr. Hlka Cayaor 

Navy taaaareb Laboratory 

Coda 7510 ■ ' 
1 Wkabingtoo, 1$ 20,373 - 

1 LT Steven D. Harrle, X5C, 0SH 
*TD 1, box 243 
tlnar, VA 2*149 

1 Or. Jim Vol lan . 
Coda 14 

r Navy Pereoooel t a 0. Center ' 
San Diego, CA 92152 

1 Dr. Id Botch! na 
Navy peraonnel HAO Canttr 
Sao Mego, CA 92152 



1 Or. 'Jot McLaehlaa 
Navy Pereoaael UD Ctnttr 
San Dtafo, CA 92152 



1 Dr. Norman J. tarr 
Qilaf of Tlaval Ttchnlcal Training 
Naval Air Station Henphle (75) 
Killing ton, TN 38054 

1 Dr. Patar Klncald 
Training Analyalt 8 Evaluation Croup 
Dept. of the Navy 
Orlando; PL 32813 

1 Dr. VI 11 Ian L. Kaloy (02) 
- - Chi a* of lUnral Vacation •odTriidlng 
* Naval Air Station 
Peaeacola, PL 3250$ 

I CAPTHiciarJ .L. Mama, DSN 
Consandlng Officer 
OSS Carl Vina on (CV3H70) 
PPO Haw Tork, Wt 09558 



1 Or. George Moeller 

Director ♦ tabavloral Selaaeaa Oept 

Naval Submarine Kedltfal taaaareb * 
' Naval Submarine taat 

Crotoo, CT 06349 

' 4.' 

1 Or Wllliaa Montague 

NPtDCJCode 13 \ 
. San Diego,, CA 1 92152 J 

\ Library, Coda P201L * ■ . 

Navy Veteonael HirCanter* 
San Diego, >.CA 9J1S2 . 

, 1 Technical Director 

•Navy PereonoU X%*> Ce^t*r 
San Dlago, a 92152 

6 Coonandlng Officer 
Naval taaaareb. Laboratory 
Coda 2627 

* Waahtngton^DC 20390 

1 Office of Neva teeeereh 
Code, 433 

800 I. Quiney SStraat * 
Arlington, VA 22217 

6 Personnel "V* Training taaaareb Croup 
Coda 442PT ^ 
Off let of Neva taaaareb t 

* Arlington^ VA22il7 

1 Offlea of tha Chief of Natal Optretlone 
taaaareb Davalopnant * Studies Branch 
OP 113 

Washington, DC 20350 

1*LT Prank C. Patho, HSC, OSS (Pb.0) 
CKET OM32) 

NA*> ^ 
FL 3250* 

Pooek 

Operations taaaareb Departnani 

Code SS?7. 

tfxval Postgraduate School 
Honteray, CA 93940 

1 Dr. tobart C. Snitb .. . 
Otflce of Chief of Neva Operations 
0P-987B 

Waablngton, DC 20350 



" Marine Corpe . 

1 H. Wljllee .Greenup ' 
Education Advleor (2031) : 
Education Center, HCDtC 
Quantleo, VA 22134 , ^ ' 

»* « • 

•• 1* Special Asslstsne for Mtrlot 
Cote* Kattere 
I *. A Code 100H 
' 'Office of Naval taeearch 

800 N. Qulncy St.' . 
I Arlington, VA 22^17 

* t Dt. A.L. SLAfEOSKI - 
\^/ SCIENTIFIC ADVISOR /(CODE tb-y 
HQ, U.S. HARINE CORPS 
WASHINGTON, DC 20380 
\ 




Any 

t Tt conic a Olreccor ? 

0. S. Aray teeeereh tnet*tu.4 f*r cba 
tebavlora and Soelal Itlencea , 
5001 tlaanhowtr Avenue 
Alexandria, VA 22335 

fi Or. tantrlca J. Parr . • 

0. S, 'Amy taeearch Xnatltute > 
3001 tlaanhqwer Avenue ■ • - x 
Alexandria. VA* 22333 

1 Dt« Harold 7. VNell, Jr. - 
Direct or , Training taeearch Lab 
Amy tuearcn lnacltuta 
3001 tieenhower Avenue 
Alexandria* VA 2213) K 

. . 

1 Connandar, O.S. Amy taaaareli Xnatltota 
for tba Saha^lora 1 Social" Sclencaa 
ATTN: PSU-S1 (Or. Judith Oraaaai) 
. 3001 Eleenhover Avenue 
Alexandria, VA 20333 . 



Pb.0. 



1 Dr. Alfred P. Sonde, Director* 
Training Analyale 6 Eva nation Croup 
Dept.' of ' tba Navy 
Orlando, PL 32813 



Vi a 



1 Or* 11 chard Sorenaea 
$*rj Parfonoa tAD Canter 
San Diego, CA 92 13 2" . 



1 Dt. Pradarlck Stelnbaleer 
cm - 0P113 
Navy Annex 
Arlington, VA 20370 % 

"1 toger Weleelnger-aaylon y 

Oapartnant of Adninletretlve Seleoeee 
Neva Poetgraduate Scbool 

„a 91140 — ^ 



Mr John H. Wolfe 

Navy Pareounel UD Canter 

San Dingo, CA 92152 

Or. Wallace Wulfeck, 121 
Navy Pareounel MO Oancer 
San Dingo, CA 92152 



a Joaeph Peotka, 
ATTN: KtX«lC 
Amy tVaeearch lnacltuta 
5001 Xleenhotfar Ave. 
Alexandria, VA .22333 

1 Or. tobart Saenor 

0. S. Any taieerch Institute for tba 
tabavloral and Social Sclaneae 
5001 daeattover Avenue 
Alexandria, VA 22333 

1 Or. tobart Wleber 

Amy X«a«arcb Iaecltute 
5001 Eleerahovar Avanua 
aaxandrla, VA 22333 



Air Porce j 

1 O.S. Air Poree Office of Scientific 
taaaareb 
Life Sclencee Directorate, T* • 
Soiling Air Poree taae 

% 'il^bington, DC 20332 

J JtCx Jaynond X. -Chrletel — — 

KfUUtOL 

lrooka Aft, IX 78233 ~ 



I tryan Dal lean 

APHtL/UT 
, Lovry AP8, CO 80230 



BEST COPY AVAILABLE 



19 



1 Or. Genevieve Bedded 
Program Manager 
Lift Sclaocaa Direccorece 
AFOSR 

BolUn* AFB..DC. 20332* . . 
• • 

1 Dr. John Tengney 
AFOSR/NL . 
polling A7B, DC; 20332 

I Dr. Joeoph Taaetuke 
AFHRL/LRT 

to wry An, CO '80230 



DiftrtBtac of Miaii * V 

12 DtftBM Technical Information Cancer 
Cameron Station, B*dg 5 
Unxaadrle, NfA 223X4 

Accn: 72 

1 Military Aaaiatanc for Training and 
Par enamel Technology 

Offlee of the Under Secrecary of Da fans 
• for Bee a arch ft Engineering 
loon 3D129. The Pentagon 
Vaaalngton, DC 20301 

1 Major Jack Thorpe 
DARFA ./ 
U0O tftleoo Blvd. 
Arlington, VA 22209 



Civilian Afancj.ai 

1 Dr k Patricia A. Buclar 
"IE-SI* Bldg, Stop # 7 
1200 19th St.. W# 
Vanning too, DC 20206 . . 

1 Dr. Sua »n Chlpnan 
Learning and Developmenc 
National Inatltuta of Education * 
1200 f9ch Straat HV 
Vanning ton, DC 20208 

1 toward Eacy 
Depertaent of Education, OEM 
MS 40 ' 
1200 19th St., NV 
Vaahlnftoo, DC 20208 

1 Dr. Arthur Malnad 
724 Brown 

0. S. Dapt. of Education 
Vanning coo, X 20208 

1 Dr. And raw X. Holnar 
Office of Scientific And thglnearlng 

Par io anal and Education 
Na clonal Science Puliation 
Vanning too, DC 20330 

1 Dr. Braratt Palmer 
lUil 1:>p 239-3 
NASA-Ami Reaeereh Cantar 

Moftttt Flail, Cv «035 

1 Dr. Mary Stoddard 

C 10, Mall Stop B29* 

L01 Alaaoa National Laboracorlae 
_ Ua Alaaoa. VI 87)45 

1 Chief , Payeholoflcnl tasarch Branch 
S. Co* at Guard (C-P-1/2/TP42) 
Veehington, DC 20393 

1 Dr. Edward C. Valaa 
••Clonal Sclanca Pound a c Ion ' 
1800 6 S crane, HV 
Vaahlnf ton,. DC 20330 



\ 



1 Dr. Prapk VI throw 
0- S. Office of Education 
*00 Maryland Ave.* SV 
.Vnahlngtoo, DC 20202 a 

1 Dr. Joaaph L. Young, Director 
Memory ft Cognitive Proceeeea 

, Hetfonnl Science Foundation 
Vafhiottoo, DC' 20330 



Private Sector* 

1 Dr.; John *. Andarioo * • 
DeparcmBoc of Payebo^Dgy ' 
Cernefle^lellou'Dolverelty 
Pitiaburgh, PA 13213 

1 Or. Patricia Befgacc 
Deportment of PiyctaU^y 
Onivereity of Colorado 

- Boulder, CO 80309 



1 Mr; Avron Barr 
Depercpent of Computer Science 
Stanford Onlvarelty 
Stanford, CA 94303 

I Dr. Menucha Blranbeun 
School of Education 
Tel Aviv Onlverllty 1 
Tel Aviv, Ram*; Aviv 69978 
Iireel 

1 Dr. John Black • ' ■ 
Tela Dnivarelty * 
Box UA, Tali Stetloo 
Mew lev an, CT 06320 

1 Dr. John S. frown 
XEROX Palo Alto Reeaarcl Cemar 
S133 Coyote Road 
Pain Alto, CA 94304 

1 Dr. Glenn Bryen 
6208 Poe Road 
'* Bcthaede, MD*20817 

1 Dr. Jaime Car bona 11 
Carnegie-Mellon tJalveraiey 
Depertaent of Peychology 
Pltteburgh, PA 13213 

1 Dr. Fat Carpenter 
Depertaent of Peychology 
Carnegle-Melloftr-Dnlverfity 
Pltteburgh, FA 13213 

f<Dr. Vtlllaa Chaae 
ifartaent of Paychology 
CerSegie Mellon Ualvereity 
r- ; Plttaburgh, PA 13213 

lf*Dr. Michelle* Chi 

' Learning la 0 Centar , 

Uulveralty of Pitteburgh 
. 3939 O'Hara Street 

Fltteborgh, PA 15211 » 

S v. luteal Cole" : 
Onlvernlty of California 

nt tan Dingo 
Laboratory of Conparacivn; 

Banana Cognition - D003A 
La Jolla, CA 92093 

1 Dr. Allan M. Colllne 

Bolt Beranek A Newman, Inc. 
30 Moulron Street 
Cambridge, MA 02138 



Dr. Lynn A. Coo par 
LR3C . '■/. \ 

Oniveralcy of Pltteburgh 
3939 O'Hara Strait \ 
Plttaburgh, PA 13213\ 



'1 ERIC Pnclllty-Acqjlflflnna 
4833 Rugby Avenue 
BetfpuU, "«o 20014 

l'profaaeor fceuveo Pauert.tel'1 
HVCaU Kabov Kara 00 6 
Bet Hekarmm 
' Jeruaalem 
Xarnel 



1 Mr. VaUacie Paurselg 

Department of Educaclooel Technology 
' Bole Baxeaak 6 Newman ' 
10 MouXroa Sc. 
Caabrltfgn, MA 02238 

1 Dr. Dexter Fletcher 
VICAT fteaaarch Inetltute 
1873 S. State Sc. 
y Oram. UT 22333^. v , . 

* } 
1 Dr. John t- Fredarlkean 

v Bait Beranak ft Newman 
30 Hmlton Street 
Cambridge, MA 02138 

1 Dr. Don^eotnar - 
Center for &aia Infomeclon PTOceaelog 
Onlverelcy of California. San Diego 
La Jolla, CA 92093 
. • 

1 Dr. Dedre Centner — ,> — r 

Bolt Bereoek ft Newuan 
■ 10 Moultoo Sc. 
Caabrldga, MA 02138 

1 Dr. -Roberc Gleeer 
Learning Reaeerch ft Developaent Center 
Onlvenalty of Pltcaburgh^ 



3939 0'Her% Scraec 
PlTTSBORCd, FA 13260 



1 Dr. M%rvln 0. Clock 
217 Scone Ball 
Cornell Uolverelcy 
lchace, NT 14833 

. % 

1 Dr. Joaph Gcfeuao 
^ SRI International 
333 Rav ana wood Avenue 
Menlo Fork, CA 94023 

1 Dr. Daniel Gopher 

Faculty of Indue c rial Engl near log 

& Management 
TtCRM10N s 
Haifa 32000 
ISRAEL 



&2E! 



M0 



1 DR. JAMES C. 
LRDC 

0N1V3RSITT OF FITTSBDRCB 
1939 0'HARA ST1S5T 
FITTSBinLG^, PA 13213 

1 Dr. Barbara RayaB-Roth 
Depertaent of Coiputer Science 
Scanford Dnivarelty 
Stanford, CA 93J05 

1 Dr. Rerl Hunt 
Dept. of Peychology 
Onlverelcy of Veahlngcoo 
Seectla, V A 98103 

1 Dr. Marcel Juet 

Depertaenc ef Peychology 
Cernegle-Mellon Onlvaralty 
Pltteburgh, FA 13213 



20 



0 

ERIC 



1 Dr. DvrU Klsras 
Dapartaaot of Psys.wUgy 
Qnlvaralty of Arltoaa 
Tuscan, AZ 83721 

I Dr. valtarJHbtseh .. ' 
Dajpartmant orPayehology 
Onlwaralcy of Colorado 
Bouldar^ CO 80302 , 

1 Dr* Stophao Kosslya« •> 
1236 William Jamas tail 
33 llrYlaad ft. 
Cambrldgm, HI 02138 * 

1 Dr. Pat Laoglar- 

Th« tobotles Instituts - 

C*rna*io— iilloo Onlvarslcy 
" Pittsburgh, PA 15213 

'1 Dr. Jill Larklo 

Dapartmant of Psychology 
Carmagla Halloo University 
Pittsburgh, PA.13213 v ~ ' — 

1 Dr. Also La t gold 
Laarnlng BAD Cantar 
Onlvaralty of Pittsburgh 
3939 O'Hara Straat 
Pittsburgh, PA 13260 

1 Dr. Jim Win ^ 
Dnl varsity of California 

sc Sob Dlogo 
Laboratory f of Comparatlva 

Human -Cognition * 30034 
U Jolla., CA 92093 

■ « 

1 Or. Kich*sl Uvlat. 

Dapartaant of Edocstlooal Psychology 
. .210 Education Bldg. 

Onlwsralty of Illlools 

Champaign, XL 61801 

1 Dr. Kjrclt C. Linn , 

Lovrsnco 8*11 ot Selsncs 
' Dalvsrolty of Calif ornls 

aarkalay. CA 94720 

1 Dr. Don Lyon • 

APHXL/OT (UMtl) *~ 

Williams API, AZ S3223 

1 Dr. Jay\ MeClalland 

DapartmW of Psychology 
MIT l 

Caabrldgl, HA 02139* 

1 Dr. J«**s ft* Miliar . 
Com puts r*Tbought Corporation 
1721 Vast Plaao Highway 
Plana , TX 73073 

I Dr. Hark Mlllsr 
. Coaputar*Thought Corporation 
:'■ 1721 Vast Plaao Parkway 
Piano,. TX 73073 

X Dr. Tom Moras 
farm PAJtC 

3333 Coyota Sill load 
. Palo Alto, CA 94304 * 

1 Dr. Allan Uaro 
BahavioraX TachnoXogy LaborattrUa 
1843 liana Awa» t Fourth PXoor 
tadooao teach, CA 90277 ' 



Dr. Donald A Sormao 
Cogalclva Selsncs, C-013 
0*1 w. of California, Ian Dlago 
La Jolla. a 92013 



1/Dr. Jssaa Orlaoa'ay 
'lostUucs for Oaf ansa Asalysss 
1601 X. Baauragard St. 
Alexandria, VA 22311 

. 1 Prof u Saynour Papart * 
20C-139 

Kasaaehusatta Inatltuc* of Tachnology 
Caubridga. HA 02139 

* 1 Or. Kaney Panning ton 
Onlvarslcy rf Chicago 
Craduata School of Baaloasa 
1101 I. 38th St. 
Chicago, XL 60637 ; r 

. 1 DA V rem POLS 0H » 

DIPT. 0T PSYCHOLOGY 4 
tmiVOLSlTT OP COLORADO 
3XLDIA, CO 80309 

1 Dr. Prad la If 
Physics DapartBsnt 
UnWsrsicy of California 
Barkalay, CA 94720 

1 Dr. Lauraa ftasnlck ' 
UDC 

JJnivarsity of Pittsburgh 
3939 O'Hara Straat 
Pltubnrgh. PA 152X _ 

1 Or. Jaff Wcbardaon ; 
Dauwar taaaarcb Instituts 
Onlsarsity of Daawnr 
Danwar, CO 80208* 

1 Mary S. lllay 
Program In Cogaltlwa Sclaoca 
Cantar for Busts Information *r Astatine 
Onivaraity of California. San Dlago 
U JoXIa. CA 92093 

1 Or. Aairav Ross 
Amarlean &scltutaa for Aassareh 
1035 Thorn** Jaff arson St. 1W 
Washington, DC 20007 

1 Or. Ern*t f . |otJhJtQpX _„....— 

salt Laboratorlss 
Murray Bill,. RJ 07974 

1 Dr. VI IX laa B. ftouaa 
Caorgia Instltuta of TachnoXogy 
School of Industrial A Syataas 
; lag 1m mrlng 
Atlanta, CA 30332 

1 Dr. Da-si-i tuaalhart 
Cantar tor ixsaa Information Procassinc 
Oniw. of California, 5«n "K*r> 
La Joiia, CA 92093 4 

I Dr. Hlchsal J. Samat 
Par cant Tonics, Inc 
6271 Vartal Amu* 
Woodland Sills, CA 9X364 

1 Dr. Bogar Schaok ^ 
Tala Univmrslzy ' - / 

Dspartmmmr of Computar Sdamca . 
P.O. Box 2138 
Nov larmn. CT 06320 

1 Dr. sTaltar Schnaldar 

Psychology Dspartmant 
' 803 k% DaalaX 

Champaign. 1L 6X820* 

1 Dr. Alan Scnoaof aid 
teebaaatlea and Sducation 
Tns DnlBaralty of %>oaatar 
tochastar, HT 146 



1 Sir. Colin Shsppard 
• Appl lad 'Psychology Ontt 

Adalralty Marina Tachnology Eat. 
Todd lAg too, Htddl'saax 
Onltad llngdoa 

1 Or* H* UalXaea Slaalko 
Program Olraetor 

Maopowr Aaaaareh and Advisory Sarvicas 
w Salthaoolan Inatltutlon % 
.801 Hortb Pitt Straat 
aiaxandrla, VA 22314- 



1 Dr. WwirdX Saith 
Bolt Bsransk 6 Sr/aLi, Inc; 
50Doultoo Strsst . ^ 
Csmbrldga. H\ 0213) 

s. r 

1 Dr. %L;aird Snou. v 
. School of Cducatloo 

Stanford OnLvarslty . 

Stanford, CA 9*305* * 

1 Or. EXlott Sol ova j 
Tala Unlvaralty 
Dapartaaot of Coapatar Sclaoca 
P.O. Box 2158 
Haw Bavan, CT 06523 

1 Or. Kathryn T. Spoahr 
PaycboXogy Dspartaant 
Brown OnlVaralty 
Provldaaca, U 02912 

1 Or. lobart Starnbsrg 
Dspc. of Psychology 
Tala Onlvaralty 
Box 11 A, Tala Station 
Slaw Hasan, CT 06520 

1 Iir. Albsrt Stsvsns 

Bolt 3ar*nn'& « Nswaan. Inc. 
10 rtoultoa Sc. 
Cambrldga, MA 02238 

1 Oavld ». Stona, Ph.D. 
Hasaltloa Corporation 
7680 Old Sprlnghouaa toad 
HeUan, VA 22\02 • 



I Or. Kitum* Tatauoka 

SS ? , tS f t * itd s -«««io« Dosaarcb Las 
" 11 En « ln *«Tlng Aaaaareh Laboratory 
Orbanm, IL 61801 

1 Dr. Parry V? Thorndyx* v , 
Parcaptronics, Inc.. 
943 MlddXaflaXd Aoaa, Snlta 140 
>Wnlo Park, CA 94023 

1 Dr. Douglas Town* 
*aiw. of So. California 
Bahawloral TachnoXogy Labs 
1843 g. tlaat Aw a. j' 
ftodoodo Baach, CA 902^7 

t Or. Kmrt Tan Laha 
Xarox PAAC 

3331 Coyota 81 XX load 
P»lo Alto, CA 94304 

1 Dr. Kalth t. W«scoort 
Parcaptronlca, Inc. 
343 IttddXafi.ld load, Suita 140 
Hsnlo Park. CA 94023 

1 William B. Whittan 
ball Labbraurlaa 
20-610 

BolmdaX, RJ 07733 " 



/ 



ERIC 



2J 



I Dr. Tfeoaat/Vlckoao ' 
DtpsrtMQtfaf Psychology 
ttm IaII 

tklvortlty «f California 
*0S Illgarda Avaaua 
loa Aa«ala», CA 90024 

X Dr. Hika VllUasa 
Bolt Baraaak t Hcwan 
10 Moult on Sc. 
Casarldga, HA 943Q4 

I Dr. Joaapb WohX 
Alphatach. Ia:. . 
2 Burl 1 of ton txacutlvt C«nt«r 
Hi Hlddlaia* Turnpike ' 
aotllagton,' MA 01903 



M ■ 



